A comparison of activation functions in multilayer neural network for predicting the production and consumption of electricity power

Predicting electricity power is an important task, which helps utilities in improving their systems’ performance terms of effectiveness, productivity, management and control. Several researches had introduced this task using three main models: engineering, statistical artificial intelligence. Based on the experiments, used intelligence models, multilayer neural networks model has proven its success predicting many evaluation datasets. However, depends mainly type activation function. Therefore, paper introduces experimental study for investigating with respect to different functions depths hidden layers. The experiments cover comparison among eleven four benchmark under examination are sigmoid, hyperbolic tangent, SoftSign, SoftPlus, ReLU, Leak Gaussian, ELU, SELU, Swish Adjust-Swish. Experimental results show that ReLU outperform counterparts all